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TITLE 

METHOD FOR DOCUMENT VIEWING 

FIELD OF THE INVENTION 

This invention is related to methods in which a client queries a 
document server in preparation for document transfer and, more particularly, 
to those methods which request document servers to transfer specified 
documents to the client in a particular format prior to individual client-side 
viewing. 

BACKGROUND OF THE INVENTION 

In the arts, many client-side devices communicate with a server either 
through a direct connection such as a phone line or a cable or via a remote 
connection, such that the end-user (operating the client-side device) can 
retrieve email, documents, pictures, and other text or pictures made available 
to the client by the server. Depending on the type or kind of connection the 
user has made with the server device, the transfer of documents from the 
server to the client can be relatively quick in those instances wherein the client 
is connected to the server through a high bandwidth connection, or can be 
painstakingly slow if, for example, the client is connected to the server through 
a low bandwidth connection. The server merely receives the request from the 
client device and then retrieves the desired pages or other documents or 
pictures and routinely queues them up for transfer to the requesting client. 
Depending on the bandwidth of the client-server communication pathway, the 
transfer of documents from a server to a remote client can take quite a long 
time which increases, for example, the risk of having the connection-link 
severed during the extended transmission time frame or leads to a poor user 
experience. 
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Today, many client-side devices are handheld devices, as illustrated in 
Figure 1 , with relatively small viewing screens for displaying documents. (All 
trademarks are owned by their respective organizations.) Users can connect 
to a server via a wireless protocol and request work-related documents and 
5 email for viewing while away from the office. Since many of these small 
handheld devices have very limited internal size, they routinely lack sufficient 
memory and/or storage room to hold all of the requested documents or 
document applications/viewers at any one moment in time. In addition, such 
handheld client-side devices can be limited in the processing power needed to 
10 convert such documents for display on the limited real estate of the handheld 
device's viewing screen. 

As such, one of the problems in the art of client-side document viewing 
involves the combination of accessing remote documents over a low 
bandwidth connection using a client-side device with relatively limited screen 
15 real estate, coupled with insufficient internal memory to store a large set of 
retrieved documents and document applications, and often lacking adequate 
computing power to convert such documents on-the-fly into another format for 
faster displaying on the user's viewing screen. Current solutions often restrict 
the amounts and/or types of documents that can be viewed (e.g. web pages 
20 with GIF files) by such remote handheld document viewing devices. What is 
U needed in the arts is a method by which a low-bandwidth client-side device 

can request from a remote server documents in a certain format. 

What is needed in the arts is a method by which such a client-side 
handheld device can take advantage of the often considerable computing 
25 power of the server to convert requested documents into a format with 
compression in order to speed up transmission times while gaining the benefit 
of having a smaller document (in terms of total bytes) to manipulate client-side 
prior to displaying on the limited real estate screen for the user. 
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SUMMARY OF THE INVENTION 

What is disclosed is a method by which a client-side handheld device 
requests a server to convert server-side documents into a compression format 
prior to transmission of said documents to the client. The server retrieves and 
converts the requested documents to a raster image that is then compressed 
5 according to attributes based on information received from the client device in 
the initial document request. Instead of having to manipulate multiple formats 
which the original documents are in and supported by the server, the client- 
side device is preferably optimized in hardware and/or software to support and 
otherwise take advantage of the requested compression format. The 
10 compressed document is then delivered to the client device, in whole or in 
part, selectively or progressively over time per individual requests prior to 
ft displaying the received data to the end-user. Depending on the requested 

Jj delivery mode, server-side documents are preferably compressed using 

m wavelet compression methods, such as the JPEG 2000 standard, known in 

suss, 1 
s = : ; 

%l is the arts. Through such a compression format, documents of sizes (in total 

L, bytes) exceeding one or more Megabytes can be compressed down to as 

SIS:? 

M small as 30 kilobytes or less. 

JT More specifically, the method of the present invention involves the first 

O step of a client side device requesting a document or a portion of a document 

.. ., 

20 from a server and the server receiving said request. Secondly, the server 
retrieves the requested document portion and converts said portion to a 
wavelet compressed image. The server sends packets of data from the 
wavelet compressed page image to the client. The client receives the 
compressed data; decompresses it and displays the decompressed document 

25 for the user. The client makes subsequent requests for more data which the 
server responds to using the wavelet compressed image data for the 
document. All these requests and responses use a protocol appropriate to 
document viewing and the features of the wavelet compression. In the 
preferred embodiment, wavelet compression uses the JPEG 2000 standard. 
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Advantages of the present invention include providing a viable method 
for a simple, lightweight universal document viewer that provides the user with 
improved end-to-end system responsiveness (low latency) when viewing 
documents. Other objects, advantages, and salient features of the invention 
5 will become apparent from the detailed description which, taken in conjunction 
with the drawings, disclose the preferred embodiments of the invention. 

Additional benefits and advantages of the present invention will be set 
forth in the description which follows, and in part will be made obvious from the 
description, or may be learned by the practice of this invention. The 
io advantages of this invention, as described herein, may be realized and 
otherwise obtained by means particularly pointed out and distinctly claimed in 
the appended claims, taken in conjunction with the accompanying drawings 
and detailed specification. 



¥ BRIEF DESCRIPTION OF THE DRAWINGS 



The preferred embodiments and other aspects of the invention will 
15 become apparent from the following detailed description of the invention when 
read in conjunction with the accompanying drawings which are provided for 
the purpose of describing embodiments of the invention and not for limiting 
same, in which: 

Figure 1 is an illustration of one such client-side hand held device 
20 known in the arts; 

Figure 2 illustrates the hand held device of Figure 1 displaying a 
combination of text and graphics with a portion thereof having been selected 
by the user; 

Figure 3 shows the portion of the text of Figure 2, selected by the user 
25 being displayed with a subset of that text again being specifically selected; and 

Figure 4 shows the selected text of Figure 3 displayed. 
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DETAILED DESCRIPTION OF THE INVENTION 

What is disclosed is a method by which a client-side handheld device 
requests server-side documents from a server which the server converts into a 
compression format prior to transmission of said documents to the client. The 
server retrieves and converts the requested documents, if they have not been 

5 previously converted, to a raster image that is then compressed according to 
attributes based on information received from the client device in the initial 
document request. Instead of the having to manipulate multiple formats which 
the original documents are in and supported by the server, the client-side 
device is preferably optimized in hardware and/or software to support and 

10 otherwise take advantage of the requested compression format. The 
compressed document is then delivered to the client device, in whole or in 
part, selectively or progressively over time per individual requests prior to 
displaying the received data to the end-user. Depending on the requested 
delivery mode, server-side documents are preferably compressed using a 

is wavelet compression method, such as the JPEG 2000 standard, known in the 
arts. Through such a compression format, documents of sizes (in total bytes) 
exceeding one or more Megs can be compressed down to as small as 30 
kilobytes or less. The method of the present invention involves the first step 
of a client side device requesting a document from a server in a particular 

20 compression format and the server receiving said request. Secondly, the 
server retrieves the requested document portion and converts said portion to a 
JPEG 2000 compressed image. The server sends as much of the JPEG 
2000 compressed page image data to the client as is needed to satisfy the 
client request. The client receives the compressed data; decompresses it and 

25 displays the decompressed document for the user. The client makes 
subsequent requests for more data which the server responds to using the 
JPEG 2000 compressed image data. All these requests and responses use a 
protocol appropriate to document viewing and the features of the JPEG 2000 
compression. 
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A user has a handheld device, as shown in Figure 1 , with limited screen 
or viewing real estate (in many instances, these handheld devices also have 
limited internal memory and computing power) and the user wishes to view 
some of his or her documents residing on a server in a remote location. For 
5 example, the user may want to look up something specific in a particular 
document, or verify that they have the correct document before printing, faxing 
or e-mailing it onward. The viewing device has limited memory, and lacks the 
computing power necessary to render on-the-fly documents from one format to 
another prior to viewing. Handheld devices have little screen real estate and 

10 typically operate over a relatively low bandwidth connection. The end-user 
would first download the entire document then open, reformat, rasterize, 
zoom, scroll and pan-all the operations you normally perform on a desktop 
system with a regular-size monitor. However, these operations typically don't 
map well to the small screens of a handheld device. 

15 Preferably, the client-side device is optimized in hardware and/or 

software to support and otherwise take advantage of a particular compression 
format such that the handheld device is merely an image viewer and, in 
particular, a wavelet-based image viewer, preferably one compliant with the 
JPEG 2000 standard. 

20 The JPEG 2000 code stream is organized into packets whose ordering 

represents one of multiple possible different groupings of the compressed data 
structures. This means that as more packets are received and decompressed, 
more of the image can be viewed. The particular grouping determines how 
the image progresses, by spatial resolution and by quality (bit resolution) being 

25 the two main progressive modes. In addition, region-of-interest coding can be 
used for that some of portions of the page image appear with higher quality 
before other parts of the page image. In addition compressed image data can 
be tiled, so that rectangular subsets of the compressed image can be 
accessed independently of other subsets. There are other benefits to using 

30 the preferred JPEG 2000 standard. 
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When the user requests a document using the user-interface (Ul) on 
the handheld client, as shown in Figure 1 , the client device makes a request to 
the server on which the document resides. The server retrieves the document 
and converts each page to a JPEG 2000-com pressed image. All pages can 
5 be converted, or just the page needed to satisfy the initial request, or the page 
requested along with the pages preceding and following it. Additional 
strategies are envisioned herein. The page images may also be cached or 
pre-computed. The server then responds to the client request with an initial 
image, typically a high decomposition level, low resolution image in a JPEG 
10 2000 code stream or a low quality image (most significant layers in a JPEG 
2000 code stream) of the requested page. The initial image would usually be 
in color, but a gray-only image could be initially sent. In the preferred 
embodiment, the client receives the decompressed data and then 
decompresses and displays it as shown in Figure 2. In some circumstances, 
15 the size of the received compressed image (in bytes) may be larger than the 
size of the image (in bytes) that is to be rendered on the display. In such 
instances the server can decompress, compute and send a portion or segment 
of the image to be displayed to the client device. Depending on transmission 
and decompression times, the user may be able to stop the progressive 
20 rendering of the document image on the display by deciding that they have 
m seen enough and that further transmission or decompression will not provide 

any more useful data. 

After the initial view, the user can then request more data from the 
current page image-more resolution, in effect zooming in; more quality to 
25 refine the viewed image; or full color-or the user can request another page. 
As shown in Figure 2, the user could also select a region of the page image, 
identify that region in a request to the server and then receive from the server 
the data that is needed to provide a higher resolution view of that region. In 
other words, the server sends data for only a portion of the image. All these 
30 requests would be satisfied by the server providing the appropriate packets 
from the JPEG 2000-compressed version of the page image that resides on 
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the server. Figure 3 shows another page of the same document in Figure 2; 
and Figure 4 shows the resulting displays when the user zooms in on a portion 
of a page. Note that the zooming is satisfied by additional data, namely 
resolution levels, in combination with data previously received. After all 
resolution levels have been received, then subsequent zooming would 
interpolate the received data. 

Since the JPEG 2000 standard supports ROI (Region-of-lnterest) 
coding, some portions of an image can be compressed with more fidelity and 
higher resolution. In addition, certain regional packets for selected portions of 
the image can be manipulated in the codestream prior to transmission such 
that they occur earlier in the codestream and thus are delivered before the 
arrival of packets for other regions of the image. JPEG 2000 ROI coding can 
be used in the initial image that the server delivers to the client so that the 
image bits initially received show more detail in the areas of the page that are 
determined to be more interesting or significant. The ROI can be defined in 
several ways: predefined for each page of the document (the ROI or the rules 
for defining it are predefined), determined from an analysis of the page when 
the JPEG 2000 image is created, or in response to an attribute in the client 
request, e.g. Tm interested in the text more than the images." 

Aside from different progressive modes realized by reordering the 
packets in an existing JPEG 2000 codestream, the present invention 
preferably would be supported by a client-server communication protocol 
which supports requests for more resolution, more quality, more color or gray, 
and others. (Note: this protocol could be modeled on the Internet Imaging 
Protocol (IIP), Version 1.0.5, but I want to be careful not to make a statement 
here that could be construed as inferring or suggesting that any patents or 
intellectual property on IIP would also apply to this invention.) The protocol 
preferably would support attributes that determine how the page image is 
compressed, e.g. to support progressive-by-resolution transmission and 
display. Also, the page image may be compressed so that the highest 
decomposition level (lowest resolution) corresponds to the display size of the 
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requesting client. Because of the packet nature of the JPEG 2000 
codestream, the client can also request an image of a specified size in kilobits, 
either to set the transmission time or to accommodate some memory 
constraints. 

5 In summary, what is disclosed are embodiments of a method by which 

client-side handheld device can take advantage of the computing power of the 
server to convert requested documents, or portions thereof, into a wavelet 
compression format in order to speed up transmission times while gaining the 
benefit of having a smaller document (in terms of total bytes) to manipulate 

10 client-side prior to displaying on the limited real estate screen of the user's 
client-side handheld device. 

While the invention is described with reference to a particular 
embodiment, this particular embodiment is intended to be illustrative, not 
limiting. Various modifications may be made without departing from the spirit 

15 and scope of the invention as defined in the amended claims. Modifications 
and alterations will occur to others upon reading and understanding this 
specification; therefore, it is intended that all such modifications and alterations 
are included insofar as they come within the scope of the appended claims or 
equivalents thereof. 
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